79 research outputs found
The Case for Learned Index Structures
Indexes are models: a B-Tree-Index can be seen as a model to map a key to the
position of a record within a sorted array, a Hash-Index as a model to map a
key to a position of a record within an unsorted array, and a BitMap-Index as a
model to indicate if a data record exists or not. In this exploratory research
paper, we start from this premise and posit that all existing index structures
can be replaced with other types of models, including deep-learning models,
which we term learned indexes. The key idea is that a model can learn the sort
order or structure of lookup keys and use this signal to effectively predict
the position or existence of records. We theoretically analyze under which
conditions learned indexes outperform traditional index structures and describe
the main challenges in designing learned index structures. Our initial results
show, that by using neural nets we are able to outperform cache-optimized
B-Trees by up to 70% in speed while saving an order-of-magnitude in memory over
several real-world data sets. More importantly though, we believe that the idea
of replacing core components of a data management system through learned models
has far reaching implications for future systems designs and that this work
just provides a glimpse of what might be possible
THE WAIT-AND-SEE OPTION IN ASCENDING PRICE AUCTIONS
Cake-cutting protocols aim at dividing a ``cake'' (i.e., a divisible
resource) and assigning the resulting portions to several players in a way that
each of the players feels to have received a ``fair'' amount of the cake. An
important notion of fairness is envy-freeness: No player wishes to switch the
portion of the cake received with another player's portion. Despite intense
efforts in the past, it is still an open question whether there is a
\emph{finite bounded} envy-free cake-cutting protocol for an arbitrary number
of players, and even for four players. We introduce the notion of degree of
guaranteed envy-freeness (DGEF) as a measure of how good a cake-cutting
protocol can approximate the ideal of envy-freeness while keeping the protocol
finite bounded (trading being disregarded). We propose a new finite bounded
proportional protocol for any number n \geq 3 of players, and show that this
protocol has a DGEF of 1 + \lceil (n^2)/2 \rceil. This is the currently best
DGEF among known finite bounded cake-cutting protocols for an arbitrary number
of players. We will make the case that improving the DGEF even further is a
tough challenge, and determine, for comparison, the DGEF of selected known
finite bounded cake-cutting protocols.Comment: 37 pages, 4 figure
Extending the definition of modularity to directed graphs with overlapping communities
Complex networks topologies present interesting and surprising properties,
such as community structures, which can be exploited to optimize communication,
to find new efficient and context-aware routing algorithms or simply to
understand the dynamics and meaning of relationships among nodes. Complex
networks are gaining more and more importance as a reference model and are a
powerful interpretation tool for many different kinds of natural, biological
and social networks, where directed relationships and contextual belonging of
nodes to many different communities is a matter of fact. This paper starts from
the definition of modularity function, given by M. Newman to evaluate the
goodness of network community decompositions, and extends it to the more
general case of directed graphs with overlapping community structures.
Interesting properties of the proposed extension are discussed, a method for
finding overlapping communities is proposed and results of its application to
benchmark case-studies are reported. We also propose a new dataset which could
be used as a reference benchmark for overlapping community structures
identification.Comment: 22 pages, 11 figure
Quantifying and identifying the overlapping community structure in networks
It has been shown that the communities of complex networks often overlap with
each other. However, there is no effective method to quantify the overlapping
community structure. In this paper, we propose a metric to address this
problem. Instead of assuming that one node can only belong to one community,
our metric assumes that a maximal clique only belongs to one community. In this
way, the overlaps between communities are allowed. To identify the overlapping
community structure, we construct a maximal clique network from the original
network, and prove that the optimization of our metric on the original network
is equivalent to the optimization of Newman's modularity on the maximal clique
network. Thus the overlapping community structure can be identified through
partitioning the maximal clique network using any modularity optimization
method. The effectiveness of our metric is demonstrated by extensive tests on
both the artificial networks and the real world networks with known community
structure. The application to the word association network also reproduces
excellent results.Comment: 9 pages, 7 figure
Overlapping Community Discovery Methods: A Survey
The detection of overlapping communities is a challenging problem which is
gaining increasing interest in recent years because of the natural attitude of
individuals, observed in real-world networks, to participate in multiple groups
at the same time. This review gives a description of the main proposals in the
field. Besides the methods designed for static networks, some new approaches
that deal with the detection of overlapping communities in networks that change
over time, are described. Methods are classified with respect to the underlying
principles guiding them to obtain a network division in groups sharing part of
their nodes. For each of them we also report, when available, computational
complexity and web site address from which it is possible to download the
software implementing the method.Comment: 20 pages, Book Chapter, appears as Social networks: Analysis and Case
Studies, A. Gunduz-Oguducu and A. S. Etaner-Uyar eds, Lecture Notes in Social
Networks, pp. 105-125, Springer,201
Stratification of the severity of critically ill patients with classification trees
<p>Abstract</p> <p>Background</p> <p>Development of three classification trees (CT) based on the CART (<it>Classification and Regression Trees</it>), CHAID (<it>Chi-Square Automatic Interaction Detection</it>) and C4.5 methodologies for the calculation of probability of hospital mortality; the comparison of the results with the APACHE II, SAPS II and MPM II-24 scores, and with a model based on multiple logistic regression (LR).</p> <p>Methods</p> <p>Retrospective study of 2864 patients. Random partition (70:30) into a Development Set (DS) n = 1808 and Validation Set (VS) n = 808. Their properties of discrimination are compared with the ROC curve (AUC CI 95%), Percent of correct classification (PCC CI 95%); and the calibration with the Calibration Curve and the Standardized Mortality Ratio (SMR CI 95%).</p> <p>Results</p> <p>CTs are produced with a different selection of variables and decision rules: CART (5 variables and 8 decision rules), CHAID (7 variables and 15 rules) and C4.5 (6 variables and 10 rules). The common variables were: inotropic therapy, Glasgow, age, (A-a)O2 gradient and antecedent of chronic illness. In VS: all the models achieved acceptable discrimination with AUC above 0.7. CT: CART (0.75(0.71-0.81)), CHAID (0.76(0.72-0.79)) and C4.5 (0.76(0.73-0.80)). PCC: CART (72(69-75)), CHAID (72(69-75)) and C4.5 (76(73-79)). Calibration (SMR) better in the CT: CART (1.04(0.95-1.31)), CHAID (1.06(0.97-1.15) and C4.5 (1.08(0.98-1.16)).</p> <p>Conclusion</p> <p>With different methodologies of CTs, trees are generated with different selection of variables and decision rules. The CTs are easy to interpret, and they stratify the risk of hospital mortality. The CTs should be taken into account for the classification of the prognosis of critically ill patients.</p
Financial time series prediction using spiking neural networks
In this paper a novel application of a particular type of spiking neural network, a Polychronous Spiking Network, was used for financial time series prediction. It is argued that the inherent temporal capabilities of this type of network are suited to non-stationary data such as this. The performance of the spiking neural network was benchmarked against three systems: two "traditional", rate-encoded, neural networks; a Multi-Layer Perceptron neural network and a Dynamic Ridge Polynomial neural network, and a standard Linear Predictor Coefficients model. For this comparison three non-stationary and noisy time series were used: IBM stock data; US/Euro exchange rate data, and the price of Brent crude oil. The experiments demonstrated favourable prediction results for the Spiking Neural Network in terms of Annualised Return and prediction error for 5-Step ahead predictions. These results were also supported by other relevant metrics such as Maximum Drawdown and Signal-To-Noise ratio. This work demonstrated the applicability of the Polychronous Spiking Network to financial data forecasting and this in turn indicates the potential of using such networks over traditional systems in difficult to manage non-stationary environments. © 2014 Reid et al
A Survey of Bayesian Statistical Approaches for Big Data
The modern era is characterised as an era of information or Big Data. This
has motivated a huge literature on new methods for extracting information and
insights from these data. A natural question is how these approaches differ
from those that were available prior to the advent of Big Data. We present a
review of published studies that present Bayesian statistical approaches
specifically for Big Data and discuss the reported and perceived benefits of
these approaches. We conclude by addressing the question of whether focusing
only on improving computational algorithms and infrastructure will be enough to
face the challenges of Big Data
- …